The present invention relates to locating and inspecting electronic components using machine vision systems, and more particularly to locating and inspecting surface mount packaged electronic components.
Machine vision systems are frequently used in contexts that require the system to capture a two dimensional image of a physical object and locate within that image some aspects or features that are to be analyzed, to determine position of the features and effect alignment or a particular orientation of the physical object. In such contexts, vision systems are typically implemented in conjunction with xe2x80x9cpick-and-placexe2x80x9d machinery that physically manipulates or picks up the physical object, moves the object past a camera of the vision system which captures the two dimensional image, and places the object in accordance with computations that determine the proper X, Y translation and rotation of the object.
Known systems employ what may be referred to as xe2x80x9cfeature basedxe2x80x9d methods which locate features, such as component leads or interconnects of an electronic device, within the captured image of the object to determine a current position and/or effect alignment of the electronic device. Current position and/or alignment of the electronic device being manipulated or inspected is effected in xe2x80x9cphysical spacexe2x80x9d, which is an absolute coordinate frame within which the object actually resides.
Current position and/or alignment of the electronic device in xe2x80x9cphysical spacexe2x80x9d is effected based on a model of the electronic device, i.e. standard of the physical object. Accordingly, implementation of feature based methods typically requires as an input to the vision system a set of model features acquired from the model, in accordance with an arbitrary coordinate system or xe2x80x9cmodel spacexe2x80x9d.
An image of the electronic device and its relevant features is acquired by the vision system, in accordance with an xe2x80x9cimage spacexe2x80x9d coordinate system, as the device is manipulated by the pick-and-place machinery. The acquired or captured image provides a set of image features (analogous to physical features) which are input to the vision system. The image is typically input as point information relative to the image space coordinate system. Knowing the two dimensional linear mapping between physical space and image space, the set of model features and the set of image features can be processed according to known algorithms to determine xe2x80x9ccorrespondencexe2x80x9d, i.e. a direct mapping of respective two dimensional spatial relationships, between model features and image features. Correspondence between model and image features, or equivalently, a two dimensional rigid transform that maps model features to their corresponding features in image space can then be used, knowing the two dimensional linear mapping between physical space and image space, to determine the spatial relationship between model space and physical space in order to position/align features of the object in physical space in accordance with model features in model space.
The practical realities of implementing vision systems with feature based methods used to locate features, such as component leads or interconnects of an electronic device, dictates that appropriate algorithms be designed and optimized for location/inspection of selected features. The efficacy of the system implemented requires that the appropriate algorithms take into consideration variables associated with the type or kind of features upon which correspondence will be based. Similarly the efficacy of the system requires that the appropriate algorithms take into consideration the physical background on which the selected features are located, among other things.
U.S. patent application Ser. No. 08/816,279 entitled METHOD AND APPARATUS FOR LOCATING AND INSPECTING A TWO DIMENSIONAL IMAGE INCLUDING CO-LINEAR FEATURES (xe2x80x9cthe ""279 applicationxe2x80x9d), filed on Mar. 13, 1997, the teachings of which are incorporated herein by reference, discloses a method and apparatus that optimizes the process of locating/aligning electronic components based on at least one set of regularly spaced co-linear features. In this process for aligning/locating surface mount electronic devices (xe2x80x9cSMDxe2x80x9d), a selected set of regularly spaced co-linear SMD component leads is used to quickly correlate a set of image points and a set of model points which are likely to correspond to each other. A one-to-one correspondence of a plurality of model points to a plurality of image points yields a single possible location for the object/device in the image, based on the image set to model set correspondence. The single possible location is checked for correctness, and if incorrect, a next possible set to set correspondence is tested until the correct location is found.
In the implementation in the ""279 application, a model is preprocessed to determine all sets of co-linear model features. An image feature finder, tailored or selected to find the specific type of component lead features, is implemented to extract information relating to component lead features in the image. A co-linear feature finding process finds lines of regularly spaced co-linear image features by identifying all groups of three points, testing each group of three points for co-linearity and frequency in accordance with the geometry of the model, and testing all other image features for membership in established lines based on co-linearity and frequency. A position hypothesis generator iteratively tests image line feature information against all possible model sets (i.e. lines of features) that it might match to establish one-to-one correspondence, which indicates that each feature of the set of co-linear image features is spatially correlated to or aligned with each feature of a set of co-linear features in the model. Two dimensional rigid transforms are used to effect a respective hypothesized position image for each set correspondence. Each hypothesized position image is passed to a verification process which tests the hypothesized position image against the captured image to see if it is a correct hypothesis, i.e. if all of the image features are present and in their proper locations.
The implementation of the ""279 application is optimized for processing a particular type of electronic device(s). More specifically, the ""279 application provides a high speed mechanism for inspecting and positioning surface mount electronic devices having at least one set of co-linear leads. The efficacy of the various elements of the implementation in the ""279 application is limited in applications that are not suited to processing information relating to sets of co-linear leads.
Implementations are known for inspecting/locating electronic components based on other than sets of regularly spaced co-linear surface mount leads. An implementation known in the art for using vision systems to inspect/locate Ball Grid Array surface mount devices is disclosed in ACCURATE PLACEMENT OF BALL GRID ARRAY PACKAGES, Deley, M. Ramsey, C. C., Quad Systems Corp., Electronic Packaging and Production, April 1996. Ball Grid Array devices (BGA) are surface mount devices in a package having an exterior mounting surface that consists of a set of solder balls arranged in a grid pattern. The solder balls, when appropriately positioned and subjected to appropriate temperatures, effect interconnections for attaching the BGA device to other electronic components, such as Printed Circuit Boards (PCBs).
To automatically align and place BGA devices on a printed circuit board, an automated vision system is used in a pick-and-place machine to determine the X, Y translations and rotation of the device with respect to a fixed reference point on a mounter of the pick-and-place machine. The referenced known BGA implementation finds the global position and orientation of the ball grid based on five balls located in each corner of the grid. The accurate position of all the balls is then determined by an extended attribute tree search.
The success of this known approach relies on the corner balls being located correctly. Locating selected balls in a grid of balls is not always an easy task to accomplish. Variability in lighting conditions leads to difficulty in discriminating balls from noise, such as may be created by reflected light, shadowing or the like in an image. Under less-than-adequate lighting, a ball can easily be missed. Similarly, pattern printing as is commonly found on the edges of a device body can confuse a feature locator into finding noise features instead of actual balls. Furthermore, given that ball grids on BGAs are of varying patterns, this known method may not work very well on devices with sparsely populated corners.
The present invention provides a method and apparatus for locating and inspecting BGA devices based on detecting the solder ball features in a two-dimensional digital image irrespective of the grid arrangement of the balls, irrespective of the number of balls, and irrespective of dimensional uniformity of the balls.
According to the invention, the ball grid array inspection and location method and apparatus includes a raw feature finding processor or feature finder which receives as inputs: model information in the form of a device description; captured image information in the form of a two dimensional image of the device (in image space) being inspected/located by a vision system working in conjunction with pick and place machinery; and calibration information comprising a two dimensional transform relating image space to physical space. The raw feature finding processor uses a feature finding algorithm such as a Hough transform or Cognex Blob Tool to find ball features (irrespective of number) and generate a list of raw features (in physical space) in the form of an X and Y location for each feature located.
An angle estimation processor receives the list of raw features from the raw feature finding processor and as additional inputs a user estimated orientation angle (i.e. a user supplied approximation of the rotation of the physical ball grid from a model position), and ball-to-ball pitch information from the model device description. The angle estimation process determines an estimated grid angle, i.e. a rough estimation of an orientation angle of the grid in physical space. In determining the estimated grid angle, the angle estimation processor generates projection data in the form of projections of a plurality of histograms representative of data points corresponding to lines of balls at multiple candidate angles. From the plurality of projections/histograms, the highest density peak projection is selected as indicative of the estimated grid angle.
An on-grid check processor receives the estimated grid angle, the list of raw features and the ball-to-ball pitch information and in effect superimposes a grid on the found features in order to eliminate noise features, merge features that were found in slightly different locations in multiple fields of view, and to verify/refine the estimated grid angle. The on-grid check processor produces a set or list of on-grid features in the form of a list of X and Y translation parameters for each feature/ball found on-grid.
A correspondence processor receives the list of on-grid features and model grid information (i.e. from the model device description) and searches for correspondence between the two dimensional array of found grid features and the two dimensional array of model grid features. The correspondence processor yields a direct mapping of respective two dimensional spatial relationships between model features or balls and found image features or balls. Correspondence between model balls and image balls, or equivalently, a two dimensional rigid transform that maps model balls to corresponding balls in image space can then be used, knowing the two dimensional linear mapping between physical space and image space, to determine the spatial relationship between model space and physical space in order to position/align balls of the device in physical space in accordance with is model balls in model space.
A least square final fit is effected as a fine alignment to refine the position of the device in terms of its translation and rotation. A minimum square error transformation is used in the least square fit to provide a more accurate actual device location. The device location can then be manipulated by pick and place hardware in order to place the device in its proper location/destination, e.g. for assembly on a PCB.
Features of the invention include provision of a BGA locating implementation that can be applied to virtually any BGA devices with a grid arrangement of balls, even if the grid is sparse, incomplete, or disconnected. The method and apparatus according to the invention is highly effective even under variable lighting conditions, and even under lighting conditions that do not allow all the balls in a grid to be detected. The feature locator implemented does not require balls to appear perfectly circular in an image, nor do they need to be identical to each other in size and brightness in order to facilitate location. A minimum workable ball size for purposes of detecting a ball feature is 3 pixels in diameter. Both bright and dark features can be located, hence white body ceramic, yellow body TAB (tape automated bonding) and dark body plastic BGA devices can all be inspected with the method and apparatus according to the invention. The mechanisms in the implementation according to the present invention enable inspection which tolerates moderate amounts of noise, including noise from circuit traces or other printed patterns on the device body.